When video is streamed over the Internet and played back through a Web browser or media player, the video is delivered in digital form. Digital video is also used when video is delivered through many broadcast services, satellite services and cable television services. Real-time videoconferencing often uses digital video, and digital video is used during video capture with most smartphones, Web cameras and other video capture devices.
Digital video can consume an extremely high amount of bits. Engineers use compression (also called source coding or source encoding) to reduce the bit rate of digital video. Compression decreases the cost of storing and transmitting video information by converting the information into a lower bit rate form. Decompression (also called decoding) reconstructs a version of the original information from the compressed form. A “codec” is an encoder/decoder system.
Over the last two decades, various video codec standards have been adopted, including the ITU-T H.261, H.262 (MPEG-2 or ISO/IEC 13818-2), H.263, H.264 (MPEG-4 AVC or ISO/IEC 14496-10), and H.265 (HEVC or ISO/IEC 23008-2) standards, the MPEG-1 (ISO/IEC 11172-2) and MPEG-4 Visual (ISO/IEC 14496-2) standards, and the SMPTE 421M standard. A video codec standard typically defines options for the syntax of an encoded video bitstream, detailing parameters in the bitstream when particular features are used in encoding and decoding. In many cases, a video codec standard also provides details about decoding operations a decoder should perform to achieve conformant results in decoding. Aside from codec standards, various proprietary codec formats (such as VP8, VP9 and other VPx formats) define other options for the syntax of an encoded video bitstream and corresponding decoding operations. In general, a codec standard or format allows an encoder made by one company to inter-operate with a decoder made by another company. Content encoded by the encoder of the first company can be successfully decoded by the decoder of the second company under normal operating conditions, so long as both companies have followed the relevant guidelines in the codec standard or format.
Some operations of a decoder are outside a codec standard or format, however. Such operations can vary depending on implementation, which means different decoders can implement the operations in different ways. In particular, different decoders may use different strategies to recover from the loss of encoded data (e.g., due to network congestion) during decoding. As a result of these differences in implementation, an encoder that uses one loss recovery strategy may encounter unexpected problems when inter-operating with a decoder that uses another loss recovery strategy. For example, during videoconferencing, content encoded with an encoder that uses one loss recovery strategy may be delivered to a decoder that uses the same loss recovery strategy and/or to a decoder that uses another loss recovery strategy. When encoded data is lost due to network congestion, the decoder that uses the same loss recovery strategy may quickly recover from the loss of encoded data, while the decoder that uses the other loss recovery strategy shows a blank screen or frozen screen for several seconds, or even fails to recover, requiring a time-consuming restart of the decoder.